Autonomous Grasping Using Novel Distance Estimator

This paper introduces a novel distance estimator using monocular vision for autonomous underwater grasping. The presented method is also applicable to topside grasping operations. The estimator is developed for robot manipulators with a monocular camera placed near the gripper. The fact that the camera is attached near the gripper makes it possible to design a method for capturing images from different positions, as the relative position change can be measured. The presented system can estimate relative distance to an object of unknown size with good precision. The manipulator applied in the presented work is the SeaArm-2, a fully electric underwater small modular manipulator. The manipulator is unique in its integrated monocular camera in the end-effector module, and its design facilitates the use of different end-effector tools. The camera is used for supervision, object detection, and tracking. The distance estimator was validated in a laboratory setting through autonomous grasping experiments. The manipulator was able to search for and find, estimate the relative distance of, grasp, and retrieve the relevant object in 12 out of 12 trials.publishedVersio

Autonomous subsea intervention (SEAVENTION)

This paper presents the main results and latest developments in a 4-year project called autonomous subsea intervention (SEAVENTION). In the project we have developed new methods for autonomous inspection, maintenance and repair (IMR) in subsea oil and gas operations with Unmanned Underwater Vehicles (UUVs). The results are also relevant for offshore wind, aquaculture and other industries. We discuss the trends and status for UUV-based IMR in the oil and gas industry and provide an overview of the state of the art in intervention with UUVs. We also present a 3-level taxonomy for UUV autonomy: mission-level, task-level and vehicle-level. To achieve robust 6D underwater pose estimation of objects for UUV intervention, we have developed marker-less approaches with input from 2D and 3D cameras, as well as marker-based approaches with associated uncertainty. We have carried out experiments with varying turbidity to evaluate full 6D pose estimates in challenging conditions. We have also devised a sensor autocalibration method for UUV localization. For intervention, we have developed methods for autonomous underwater grasping and a novel vision-based distance estimator. For high-level task planning, we have evaluated two frameworks for automated planning and acting (AI planning). We have implemented AI planning for subsea inspection scenarios which have been analyzed and formulated in collaboration with the industry partners. One of the frameworks, called T-REX demonstrates a reactive behavior to the dynamic and potentially uncertain nature of subsea operations. We have also presented an architecture for comparing and choosing between mission plans when new mission goals are introduced.publishedVersio

Transfer Learning in Underwater Operations

This paper investigates a method for reducing the reality gap that occurs when applying simulated data in training for vision-based operations in a subsea environment. The distinction in knowledge in the simulated and real domains is denoted the reality gap. The objective of the presented work is to adapt and test a method for transferring knowledge obtained in a simulated environment into the real environment. The main method in focus is the machine learning framework CycleGAN, mapping desired features in order to recreate environments. The overall goal is to enable a framework trained in a simulated environment to recognize the desired features when applied in the real world. The performance of the learning transfer is measured by the ability to recreate the different environments from new test data. The obtained results demonstrates that the CycleGAN framework is able to map features characteristic for an underwater environment presented with the unlabeled datasets. Evaluation metrics, such as Average precision (AP) or FCN-score can be used to further evaluate the results. Moreover, this requires labeled data, which provides additional development of the current datasets

Monocular vision-based gripping of objects

Optics-based systems may provide high spatial and temporal resolution for close range object detection in underwater environments. By using a monocular camera on a low cost underwater vehicle manipulator system, objects can be tracked by the vehicle and handled by the manipulator. In this paper, a monocular camera is used to detect an object of interest through object detection. Spatial features of the object are extracted, and a dynamic positioning system is designed for the underwater vehicle in order for it to maintain a desired position relative to the object. A manipulator mounted under the vehicle is used to retrieve the object through a developed kinematic control system. Experimental tests verify the proposed methodology. A stability analysis proves asymptotic stability properties for the chosen sliding mode controller and exponential stability for the task error